This invention pertains to solid-state image-pickup devices (SSIPDs) with motion-detection capabilities and to associated drive methods. In particular, this invention pertains to SSIPDs that simultaneously output an image signal and a motion-detection signal.
Solid-state image-pickup devices (SSIPDs) are typically used in electronic camera equipment such as camcorders, digital still cameras, and monitoring devices. SSIPDs measure light intensity at discreet locations to image a scene, and contain an array of pixels that convert light intensity into measurable voltage signals. These voltage signals are then processed to produce an output signal that may be stored or viewed on a video display.
It is sometimes desired to add a motion-detection capability to an SSIPD. Conventional motion-detection devices of this type typically detect motion by comparing the difference between frames of image data output by the SSIPD. A xe2x80x9cframexe2x80x9d of image data comprises the output signals of all of the pixels in the array during the most recent output cycle. Conventional devices of this type typically provide frame-update rates of five or more frames per second.
FIG. 9 shows the major functional blocks of a motion-detection image-processing sequence commonly employed in a conventional motion-detection image-processing device 100. The motion-detection image-processing device 100 comprises an SSIPD 101, an analog-to-digital (A/D) converter 102 that converts the analog image signal output by the SSIPD 101 into a digital signal, a first-frame image memory 103, a second-frame image memory 104, and an image-processing circuit 105 that detects motion by comparing the digital image data stored in the first-frame and second-frame image memories 103 and 104.
The motion-detection image-processing device 100 processes image data in the following sequence. During a first step, the analog image signals output by the pixels of the device during a first frame of image output are converted into corresponding digital signals (i.e., xe2x80x9cdigitizedxe2x80x9d)by the A/D converter 102. The digital signals are stored in the first-frame image memory 103. Next, the image signals obtained during a second frame, immediately subsequent to the first frame, are digitized by the A/D converter 102, and stored in the second-frame image memory 104.
In the image-processing circuit 105, the digital signal stored in the first-frame image memory 103 and the digital signal stored in the second-frame image memory 104 are compared on a pixel-by-pixel basis to detect motion. For example, all of the digitized pixel-output values stored in the first image memory 103 may be subtracted from respective digitized pixel-output values stored in the second-image memory 104; if the difference between any two corresponding pixel outputs exceeds a predetermined threshold, the pixel-output difference data may be stored in the image-processing circuit 105. By comparing the frame images in this manner, it is possible to detect motion of a subject within the image being exposed.
There are several problems with the foregoing conventional approach to detecting motion using a SSIPD. The additional circuitry required for the first-frame and second-frame image memories 103, 104, and the image-processing circuit 105 increase the size of the device, making it more costly to manufacture. Also, A/D conversion normally causes a loss in signal quality. Because the pixels are arranged in a tightly packed array, the A/D converters must be located externally to the pixel array. The analog signals output by the pixels are easily affected by peripheral noise caused by high-frequency switching of thousands of MOS switches that are in close proximity to electrical pathways connecting each pixel output to the A/D converters. Thus, by the time the pixel-output signals reach the A/D converters, the signals typically no longer accurately reflect the respective signal values at the pixel outputs.
Moreover, in a conventional motion-detection image-processing device 100, the dynamic range (bandwidth) of the image signal from a given pixel is limited by the respective A/D converter 102. Normally, the bandwidth of an A/D converter 102 is narrower than the bandwidth of the SSIPD 101. Consequently, the entire bandwidth of the SSIPD 101 cannot be effectively used for motion detection.
Conventional analog-to-digital processing is also subject to phase-shift errors that adversely affect the accuracy of motion detection. Each A/D converter 102 processes analog signals on a sequential basis in which the outputs from the pixels in a given horizontal row are processed before proceeding to the pixels in the next horizontal row. As a result, if the A/D conversion circuitry is not properly synchronized with the readout of the pixel outputs, the location of the data for a particular pixel (or sets of adjacent pixels) may be xe2x80x9cshiftedxe2x80x9d in the first-frame or second-frame image memories. For example, suppose that the digitized pixel-output data for all of the pixels in the lower half of the pixel array produced during a first frame is shifted (out of phase) relative to the corresponding digitized pixel-output data produced during a subsequent second frame. In such a case, the difference of the output data produced between frames at a given pixel location in the lower half of the pixel array can no longer be accurately measured because the data corresponding to a particular pixel location in the first-frame image memory is shifted relative to the data of the particular pixel in the second-frame image memory. This phase shift reduces the accuracy of the motion-detection device.
A potential solution to the foregoing problems, which has been considered, is to store the image signals in digital form from the first and second frames in a memory, and route the first-frame and second-frame image signals for each pixel from the memory through a comparator to measure the difference between the frames on a pixel-by-pixel basis. Such a scheme could be implemented by placing local storage circuitry and a comparator in close proximity to each pixel, or by including storage and comparator circuitry with each pixel. The problem with these schemes is that the solid-state surface area required for each such pixel and its associated memory and comparator circuitry is increased, resulting in a corresponding decrease in resolution and/or aperture ratio of the SSIPD. Another problem is that only a motion-detection signal is produced without simultaneously outputting an image signal.
In addition, frame-by-frame comparison techniques as used in conventional motion-detection devices do not accurately detect the motion of a rapidly-moving body.
In view of the foregoing shortcomings of conventional devices, an object of the invention is to provide motion-detection solid-state image-pickup devices (SSIPDs) that provide an electronic shutter function without requiring external image-comparison processing for motion detection. Another object of the invention is to provide motion-detection SSIPDs capable of simultaneously outputting motion-detection signals and image signals. Yet another object is to provide motion-detection SSIPDs that can output a high-quality image signal from which xe2x80x9cdarkxe2x80x9d signals have been removed. A further object is to provide motion-detection SSIPDs that can evaluate the motion of a body being imaged.
The invention is exemplified herein by several example embodiments that accomplish the foregoing objects by providing image-processing and motion-detection circuitry that simultaneously output an image signal and a motion-detection signal. The motion-detection circuitry compares the pixel outputs from a current frame and a previous frame, to determine if any motion has occurred between the frames, on a pixel-by-pixel basis. The image-processing circuitry subtracts xe2x80x9cdarkxe2x80x9d signals from the pixel-output signals so as to output an image signal from which the dark signals have been removed.
According to one aspect of the invention, SSIPDs with motion-detection capability are provided that comprise multiple pixels arranged in an array of at least one column and at least one row (typically an array of multiple rows and columns). Each pixel produces an electrical output signal according to a corresponding light quantity received by the pixel.
According to a first representative embodiment, a respective individual vertical-readout line is provided for each column of pixels, wherein each pixel in the respective column has a output that is connected to the respective vertical-readout line. Each vertical-readout line has an output terminus. A first vertical-scanning circuit controllably switches the outputs of the pixels in each column to the respective vertical-readout line according to a predetermined readout sequence. The output termini of the vertical-readout lines are connected to respective differential-processing circuits (i.e., preferably one differential-processing circuit per vertical-readout line).
Each differential-processing circuit receives pixel-output signals and corresponding pixel dark signals carried by the respective vertical-readout line, and provides an output signal, from which the pixel dark signals have been removed, at an output terminus of the respective differential-processing circuit. A first horizontal-readout line is commonly connected to the output termini of the differential-processing circuits. The output terminus of each vertical-readout line is also connected to a respective body-motion-detection circuit (i.e., preferably one body-motion-detection circuit per respective vertical-readout line). The differential-processing circuits store pixel dark signals delivered thereto by the respective vertical-readout lines.
The body-motion-detection circuits receive pixel-output signals obtained during a previous frame from the respective vertical-readout lines and store the previous-frame pixel-output signals. Subsequently, pixel-output signals produced during the current frame are output to the respective vertical-readout lines. A horizontal-scanning circuit provides control signals to the differential-processing circuits so that the current-frame pixel-output signals are received by the differential-processing circuits, processed, and output to the first horizontal-readout line in a horizontal-line-readout sequence to form an image signal.
Similarly, the horizontal-scanning circuit provides control signals to the body-motion-detection circuits so that the current-frame pixel-output signals are received by the body-motion-detection circuits, processed, and output to a second horizontal-readout line in a horizontal-line-readout sequence to form a motion-detection signal. The output signals of the differential-processing circuits represent pixel-output signals with dark signals removed. The outputs of the body-motion-detection circuits represent a motion-detection signal comprising the difference between the current-frame pixel-output signal and the previous-frame pixel-output signal. The device is preferably driven such that the image signal and the motion-detection signal are synchronized.
Each pixel of the first representative embodiment preferably comprises a respective MOS switch (or analogous component) for discharging the respective photodiode of the pixel and thus performing an xe2x80x9celectronic shutterxe2x80x9d function. To such end, each such switch controllably discharges residual charges in the respective photodiode in preparation for outputting an image from a current frame.
According to a second representative embodiment, a solid-state image-pickup device is provided that comprises a set of differential-detection circuits in place of the body-motion-detection circuits of the first representative embodiment. The differential-detection circuits preferably comprise multiple sample-and-hold circuits and a comparator circuit. The sample-and-hold circuits preferably comprise first and second sample-and-hold circuits that hold pixel-output signals from the current frame and from the previous frame. The comparator circuit preferably comprises a pair of comparators that receive the pixel-output signals from the sample-and-hold circuits and output a binary (digital) signal based on whether the absolute value of the difference between the pixel-output signals for the current frame and the pixel-output signals for the previous frame exceed a predetermined threshold value.
In an alternative configuration the comparator circuit comprises multiple inverters and a feedback loop. The outputs of the differential-detection circuits are read in by a horizontal-scanning circuit, preferably comprising a shift register. The horizontal-scanning circuit outputs the binary signals in a horizontal-line-readout sequence to form a single-bit digitized motion-detection signal. The device simultaneously outputs synchronized motion-detection signal and image signals.
Each of the pixels preferably comprises a respective photodiode for performing photoelectric conversion of a light quantity received by the pixel, a respective JFET (junction-type field-effect transistor) for amplifying the charge accumulated in the respective photodiode during the current frame and for storing a pixel-output signal accumulated during a previous frame. Each pixel also preferably comprises a respective MOS switch (or analogous component) for discharging the photodiode so as to perform an electronic shutter function with respect to the pixel, a respective MOS switch for shifting the charge accumulated in the respective photodiode to the gate electrode of the respective JFET, a respective MOS switch for discharging the photodiode output charge of a previous frame that is stored in the JFET, thereby performing a reset function, and a respective MOS switch to connect the output of the JFET to a respective vertical-readout line.
Each differential-processing circuit preferably comprises a capacitor and a pair of MOS switches. One side of the capacitor is tied to a respective vertical-readout line. The other side of the capacitor is tied to a reference potential through one of the MOS switches, and to a horizontal-readout line through the other MOS switch.